Google，Amazon和Microsoft等提供商提供的商业ML API已在许多应用程序中大大简化了ML的采用。许多公司和学者都为使用ML API用于对象检测，OCR和情感分析等任务。处理相同任务的不同ML API可能具有非常异构的性能。此外，API的基础模型也随着时间的推移而发展。随着ML API迅速成为一个有价值的市场，并且是消耗机器学习的广泛方式，因此系统地研究和比较不同的API并表征API随时间变化的方式至关重要。但是，由于缺乏数据，目前该主题目前没有被忽视。在本文中，我们介绍了HAPI（API的历史），该数据集由1,761,417个商业ML API应用程序（涉及来自亚马逊，Google，IBM，Microsoft和其他提供商的API），包括图像标签，文本识别和文本识别和文本识别和文本，从2020年到2022年的挖掘。每个实例都由API的查询输入（例如图像或文本）以及API的输出预测/注释和置信分数组成。 HAPI是ML API使用情况的第一个大型数据集，并且是研究ML-AS-A-Service（MLAAS）的独特资源。作为HAPI启用的分析类型的示例，我们表明ML API的性能会随着时间的流逝而大幅变化 - 在特定基准数据集上删除了几个API的精度。即使API的汇总性能保持稳定，其误差模式也可以在2020年至2022年之间在不同的数据子类型中转移。这种更改可能会大大影响使用某些ML API作为组件的整个分析管道。随着时间的流逝，我们进一步使用HAPI研究人口亚组的商业API绩效差异。 HAPI可以刺激MLAA的不断发展领域的更多研究。

部署的机器学习（ML）模型经常遇到与培训数据不同的新用户数据。因此，估计给定模型在新数据上的性能是朝着可靠的ML应用程序迈出的重要一步。但是，这是非常具有挑战性的，因为数据分布可以以灵活的方式变化，并且我们可能没有新数据上的任何标签，这在监视设置时通常是这种情况。在本文中，我们提出了一种新的分配移位模型，即稀疏关节移位（SJS），该模型考虑了标签和一些特征的关节移位。这统一并概括了几种现有的偏移模型，包括标签移位和稀疏协变量移位，仅考虑边际特征或标签分布位移。我们描述了SJS可识别的数学条件。我们进一步提出了See，这是一个算法框架，以表征SJS下的分布变化，并估计模型在没有任何标签的新数据上的性能。我们在具有各种ML模型的几个现实世界数据集上进行了广泛的实验。在不同的数据集和分配变化中，看到对现有方法的误差改善（最多达到数量级）的显着（最多）。

多标签分类任务（例如OCR和多对象识别）是不断发展的机器学习作为服务行业的主要重点。尽管有许多多标签预测API可用，但由于这些API的价格和性能的异质性，用户可以决定用于自己的数据和预算的API挑战。最近的工作显示了如何从单标签预测API中进行选择。但是，先前方法的计算复杂性在标签数量中是指数的，因此不适合像OCR这样的设置。在这项工作中，我们提出了FrugalMCT，这是一个原则性的框架，可自适应地选择以在线方式使用不同数据的API，同时尊重用户的预算。 API选择问题是作为整数线性程序施放的，我们显示的具有特殊的结构，我们利用具有强大性能保证的高效在线API选择器。我们使用Google，Microsoft，Amazon，IBM，Tencent和其他提供商的ML API进行系统实验，以包括多标签图像分类，场景文本识别和命名实体识别。在各种任务中，FrugalMCT在匹配最佳单个API的准确性的同时，可以实现超过90％的成本降低，或者在匹配最佳API成本的同时，高达8％的精度。

Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.

We propose a distributionally robust return-risk model for Markov decision processes (MDPs) under risk and reward ambiguity. The proposed model optimizes the weighted average of mean and percentile performances, and it covers the distributionally robust MDPs and the distributionally robust chance-constrained MDPs (both under reward ambiguity) as special cases. By considering that the unknown reward distribution lies in a Wasserstein ambiguity set, we derive the tractable reformulation for our model. In particular, we show that that the return-risk model can also account for risk from uncertain transition kernel when one only seeks deterministic policies, and that a distributionally robust MDP under the percentile criterion can be reformulated as its nominal counterpart at an adjusted risk level. A scalable first-order algorithm is designed to solve large-scale problems, and we demonstrate the advantages of our proposed model and algorithm through numerical experiments.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via https://mea-lab-421.github.io.